The goal of this PR is to increase the expressivity of the benchmarks interface, to allow for easy implmentations of stochastic and dynamic benchmark problems.

New abstract types

For this, we define two new sub abstract types of AbstractBenchmark:

AbstractDynamicBenchmark{exogenous} <: AbstractStochasticBenchmark{exogenous} <: AbstractBenchmark

Remarks:

• This assumes any dynamic problem is also stochastic.
• The two new types are parameterized by the exogenous parameter, a boolean indicating if the benchmark is exogenous or not (i.e. if one can generate scenarios and anticipative decisions).

New interface methods

For AbstractBenchmark

Implementing the full generate_dataset method can in practice be a bit tedious and verbose for no reason. While keeping the option to implement it for some applications where it makes sense, we propose an alternative where the user only needs to implement a generate_sample method that outputs a single DataSample instead of a full dataset:

generate_sample(::AbstractBenchmark, rng::AbstractRNG; kwargs...) -> DataSample

For AbstractStochasticBenchmark{true}

We need two new methods, one for generating scenarios, and one for computing anticipative decisions:

generate_scenario(::AbstractStochasticBenchmark{true}, instance, instance; kwargs...)
generate_anticipative_solution(::AbstractStochasticBenchmark{true}, instance, scenario; kwargs...)

For AbstractDynamicBenchmark

For dynamic problems, we additionally need to instantiate environments from instances. For this, we define the generate_environment and generate_environments methods. Similarly to generate_sample and generate_datasets, one may only need to implement the first one, that takes as inpout an instance and outputs an environment.

AbstractEnvironment

This environment type can be defined as a subtype of AbstractEnvironment, along with its associated methods:

• get_seed
• is_terminated
• step!
• observe
• reset!

Additionally, we may want to apply RL algorithms to non-dynamic benchmarks, by considering them as dynamic problems with time horizon of length 1. We therefore propose a StaticEnv <: AbstractEnv wrapper around static instances for this (#35)

Policy

We also have a new generate_policies method

generate_policies(::AbstractBenchmark)

which outputs a tuple of Policy, i.e. policies to benchmark against.
In particular you can use run_policy! to run the policy on a given set of environments for a set number of episodes each.

New Benchmark problems

• Dynamic Vehicle scheduling problem: simple example of an exogenous dynamic benchmark
• Dynamic Assortment problem: simple example of an endogenous benchmark
  We plan to additionally implement the Dynamic VRP as well as the dynamic Inventory routing.

Points I'm unsure about

• for implementing mirror descent algorithms on single stage stochastic benchmarks we will need additional maximizers, not sure where they belong in this interface
• seed management
• do we need to separate instance from environment?

TODO:

• more tests
• document new stuff